Exact distributions for reward functions on semi-Markov and Markov additive processes

Exact Distributions for Reward Functions on Semi-markov and Markov Additive Processes

The distribution theory for reward functions on semi-Markov processes has been of interest since the early 1960s. The relevant asymptotic distribution theory has been satisfactorily developed. On the other hand, it has been noticed that it is difficult to find exact distribution results which lead to the effective computation of such distributions. Note that there is no satisfactory exact distr...

Eliciting Additive Reward Functions for Markov Decision Processes

Specifying the reward function of a Markov decision process (MDP) can be demanding, requiring human assessment of the precise quality of, and tradeoffs among, various states and actions. However, reward functions often possess considerable structure which can be leveraged to streamline their specification. We develop new, decisiontheoretically sound heuristics for eliciting rewards for factored...

The KTH Visit in Semi-Markov Processes

On mean reward variance in semi-Markov processes

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for the embedded Markov chain of semi-Markov processes. Under the assumption that the chain is aperiodic and contains a single class of recurrent states recursive formulae for the variance are obtained which show that the variance growth rate is asymptotically linear in time. Expression...

Distributions of Reward Functions on Continuous-time Markov Chains

We develop algorithms for the computation of the distribution of the total reward accrued during [0, t) in a finite continuous-parameter Markov chain. During sojourns, the reward grows linearly at a rate depending on the state visited. At transitions, there can be instantaneous rewards whose values depend on the states involved in the transition. For moderate values of t, the reward distributio...